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ABSTRACT 



The Third International Mathematics and Science Study 
(TIMSS) provides extensive comparisons of cross -national rankings in 
achievement as well as rich, descriptive analyses of the students, teachers, 
and schools participating in the study. In spite of the wealth of achievement 
and descriptive data available, the two are rarely linked in research. In 
this seventh issue of "Policy Issues," an empirical link is provided between 
the performance of eighth-grade mathematics students and the educational 
practices of more rigorous curricula and conceptual approaches to 
instruction. Relationships between achievement and educational practices, 
previously based on theories of best practices, are now substantiated with 
data. In this regard, the North Central Regional Educational Laboratory 
(NCREL) takes a significant step forward in using large-scale assessment data 
like TIMSS to help state and local policymakers make good decisions to 
support exemplary math performance. After providing some background, this 
article discusses how economic status affects achievement and how schools 
contribute to achievement. (Contains 11 references.) (ASK) 
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Alboett This Issm 

The Third International Mathematics and Science Study (TIMSS) provides extensive comparisons of 
cross-national rankings in achievement^ as well as rich descriptive analyses of the students, teachers, 
and schools participating in the TIMSS. In spite of the wealth of achievement and descriptive data 
available, the two are rarely linked in research. 

In this edition (^/Policy Issues, an empirical link is provided between the performance of eighth-grade 
mathematics students and the educational practices of more rigorous curricula and conceptual 
approaches to instruction. Relationships between achievement and educational practices, previously 
based on theories of best practices, are now substantiated with data. In this regard, the North Central 
Regional Educational Laboratory (NCREL) takes a significant step forward in using large-scale 
assessment data like TIMSS to help state and local policymakers make good decisions to support 
exemplary math performance. 



New Evidence Links Curricula and 
Instruction to Mathematics Achievement 



By Blase Masini and Judy Taylor 

Introduction 

T ]he poor performance of U.S. students on the 

Third International Mathematics and Science 

Study (TIMSS) has received considerable atten- 
tion from both educators and the general public. 
Also gamering attention has been the outstanding 
TIMSS performance of the First in the World 
Consortium (FITW), a group of suburban Chicago 
school districts that independently administered 
the TIMSS to its students, with the results ranking 
among the best in the world. Located on 
Chicago’s North Shore, FITW communities have 
substantial tax bases, high per-capita income, and 
high levels of educational attainment. 

Districts that are not as well endowed often use 
these socioeconomic advantages to explain the 



high achievement levels of FITW districts, and 
others like them, as the consequence of blessings 
received and not effort expended. FITW admin- 
istrators and teachers instead point to excellent 
curricula, professional development, and innova- 
tive leadership as possible reasons for exceptional 
achievement. 
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Such contrary opinions generate 
questions for policymakers. What 
really accounts for the difference 
between the TIMSS performance of 
U.S. students and FITW students? 

Do these factors translate into policy? 
If the contribution of educational 
practices is significant after account- 
ing for students’ economic status, 
then perhaps there are lessons to be 
learned from this ambitious consor- 
tium of FITW districts. 

In this edition of Policy Issues, pohcy- 
makers will find evidence of a link 
between good educational prac- 
tices— specifically in curricula and 
instruction— and math performance. 
In addition, the North Central 
Regional Educational Laboratory 
(NCREL) isolates the contribution 
of socioeconomic status (SES) to 
student achievement to allow for a 
more accurate look at the contribu- 
tion of curriculum and practice. 
Critics often dismiss gains made by 
students from affluent families as a 
natural outgrowth of their economic 
advantage. NCREL’s analysis con- 
firms the contribution of SES on 
achievement, but it also provides 
substantial evidence that factors 
beyond SES also contribute to a 
student’s performance. 

Some Background 

The TIMSS was an assessment 
administered in 1995 in 30 languages 
to more than 500,000 students in 41 
countries. TIMSS measured and 
compared students’ mathematics 
and science knowledge and skills in 
Grades 4, 8, and 12. TIMSS also pro- 
vided educators and policymakers 
with important contextual informa- 
tion from curriculum analyses and 
from teacher and student surveys. 



Much has been written on the results 
from the TIMSS achievement test in 
recent years. Descriptions of the edu- 
cational practices of the classrooms 
and schools participating in TIMSS 
also have been detailed. For example, 
the National Center for Education 
Statistics (NCES) has published three 
editions of Pursuing Excellence, one 
each for Grades 4, 8, and 12 (U.S. 
Department of Education, 1997, 

1996, and 1998, respectively). Each 
edition provides a synthesis of initial 
findings from TIMSS to inform edu- 
cators, parents, policymakers, and 
other education stakeholders on the 
strengths and weaknesses of U.S. 
schools from an international perspec- 
tive. Separate chapters are devoted 
to mathematics and science achieve- 
ment, mathematics and science cur- 
ricula, methods of instruction, and 
teacher and student background. 

In the TIMSS pubhcation, A Splintered 
Vision, Schmidt et al. (1997) present 
substantial data on the unfocused state 
of the U.S. mathematics and science 
curricula. They conclude that poor stu- 
dent performance is likely a result of 
unfocused curricula. 



FITW districts successfully peti- 
tioned the U.S. Department of 
Education for the right to administer 
the TIMSS to their fourth-, eighth-, 
and twelfth-grade students. After 
completing this study, FITW forged a 
partnership with NCREL to learn 
how to use their TIMSS data to 
improve local practice. 

A First Look at What We Can Learn 
From High-Performing School 
Districts (U.S. Department of 
Education, 1999) provides a detailed 
report of student achievement and 
teacher practices within the FITW. 
First Look also provides the first 
comprehensive overview of FITW 
student performance and an initial 
exploration of some of the reasons 
for their performance. 

How Ecoeomic 

Status Affects Achievement 

In this analysis of TIMSS data for 
the NCREL study, NCREL com- 
pared the average classroom perfor- 
mance in eighth-grade mathematics 
achievement for U.S. classrooms 
(mean = 497) and FITW classrooms 
(mean = 591). Average classroom 
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Table 2 


Average Difference 
in Classroom Math Score 


FITW Advantage 


106 


Attributed to Economic Status 


63 


Attributed to Other Factors 


43 



scores on the TIMSS mathematics 
test ranged from 370 to 692. The 
complete distributions for the U.S. 
and FITW are presented in Figure 1 . 

Factors associated with economic 
advantage also were considered. 
These included both student factors, 
such as the number of books in stu- 
dents’ homes, and school factors, 
such as per-pupil expenditure. (A 
complete list of economic factors is 
presented in Table 1.) On average, 
FITW classroom achievement on the 
TIMSS mathematics test was 106 
points higher than that of U.S. class- 
rooms (Table 2). Economic factors 
accounted for 63 of the 106-point 
advantage enjoyed by FITW class- 
rooms. But with 43 points remaining 
in the difference, clearly something 
beyond socioeconomic advantages is 
at play in FITW classrooms. 



Table 1 

Measures of Socioeconomic Status 

Student Level: 

Number of books in student’s 
home 

Race/ethnicity of student 
Non-English language 
spoken in the home 

Parental education 
Classroom Level: 

Per capita income 

Percent of students eligible 
to receive free lunch 

Per pupil expenditure 



How Schools Contribute 
to Achievement 

Analysis of TIMSS data often 
implies a strong relationship between 
educational practices and student 
achievement. Indeed, most educa- 
tional stakeholders hope that educa- 



tional practices are related to 
achievement. As early as September 
1997, NCREL offered a set of rec- 
ommendations to policymakers 
based on its initial analysis of 
TIMSS data. In June 1999, NCREL 
published a second Policy Issues 
devoted to TIMSS analysis in which 
NCREL argued that educational 
practices of high-achieving schools 
could serve as benchmarks toward 
which others could strive. 

Yet it is only more recently that 
NCREL has confirmed the relation- 
ship between instructional practices 
and high achievement through empir- 
ical analyses. Several curriculum and 
instructional measures created from 
sets of teacher survey items were 
related significantly to mathematics 
achievement. (Table 3 provides a 
sample of survey items used to create 
each measure.) Furthermore, more 
sophisticated analyses revealed that 
all measures still were related signifi- 
cantly to mathematics achievement 
after taking into account the influence 
of economic status. A full report is 
available at www.ncrel.org/re/ses. 

FITW and U.S. 

Classrooms: A Comparison 

This research presents strong corre- 
lations between certain educational 
practices and achievement after tak- 
ing SES into account. The correla- 
tions between these practices and 
average classroom achievement exist 



in all classrooms in this study, both 
FITW and U.S. Therefore a final 
question remains: Are these exem- 
plary practices present more often in 
the FITW classrooms than in U.S. 
classrooms? 

A prior descriptive study on FITW 
classrooms indicates that the prac- 
tices associated with higher achieve- 
ment in this study indeed are present 
in many of the FITW classrooms. 
This point is emphasized only as a 
means of sharing with others 
“nuggets” of good practice. 

Curriculum 

Among the curriculum variables, 
the percent of topics covered prior 
to eighth grade was related positively 
to mathematics achievement. In con- 
trast, the percent of topics new to 
the eighth-grade curriculum was 
related negatively to mathematics 
achievement. First Look details how 
curricular content plays out in FITW 
and U.S. classrooms. 

Of note, geometry and algebra topics 
are taught earlier in more FITW 
classrooms than U.S. classrooms. 
These same topics are presented 
more often as new material in U.S. 
classrooms (i.e., content not covered 
prior to eighth grade). In short, FITW 
teachers tend to teach in a manner 
associated with higher achievement 
by introducing more sophisticated 
mathematical concepts earlier in stu- 
dents’ academic careers. Recent 
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Table 3 

Sample Teacher Survey Items Incorporated Into 
Instructional Practice and Curriculum Measures 

Instructional Practices 
Importance and Amount of Drilling 

To be good at mathematics at school, how important do you think it is for 
students to remember formulas and procedures? 

To what degree do you agree or disagree with the following statement: If students 
are having difficulty, an effective approach is to give them more practice by 
themselves during class. 

To what degree do you agree or disagree with the following statement: Mathematics 
should be learned as sets of algorithms or rules that cover all possibilities. 
Understanding Concepts 

In your mathematics lessons, how often do you usually ask students to explain 
the reasoning behind an idea? 

To be good at mathematics at school, how important do you think it is for 
students to understand mathematical concepts, principles, and strategies? 

To be good at mathematics at school, how important do you think it is for 
students to be able to provide reasons to support their solutions? 

Utilization of Textbook in Teaching 

When planning mathematics lessons, how much do you rely on student textbooks? 

When planning mathematics lessons, how much do you rely on other textbooks 
or resources? 

Approximately what percentage of your weekly mathematics teaching time is 
based on the text? 

Curriculum 

Percent of Topics Covered Previously 

The percent of topics covered prior to Grade 8 from a list of 44 mathematics topics. 
Percent of Topics New 

The percent of topics introduced for the first time in Grade 8 from a list of 
44 mathematics topics. 



research has shown that this pattern 
of covering algebra in middle school 
is strongly associated with achieve- 
ment growth throughout high school. 
These positive results are consistent 
for all students, regardless of math 
skills prior to entering high school 
(Gamoran & Hannigan, 2000). 

Instruction 

Among the instructional practices in 
this study, practices that involve 
drilling students were related nega- 
tively to mathematics achievement 
while practices that promote concep- 
tual understanding of material were 



related positively to mathematics 
achievement. These findings provide 
an interesting contrast to prior 
research (NCREL, in press). 

Both U.S. and FITW teachers believe 
that mathematics is best understood 
when students foster conceptual 
understanding by generalizing, syn- 
thesizing, and evaluating mathemati- 
cal concepts rather than follow a 
structured set of steps and perform 
such tasks as algorithm drills. 
However, FITW teachers stress more 
conceptual understanding than U.S. 
teachers. In First Look, more FITW 
than U.S. teachers ask students to 



explain the reasoning behind an idea 
and write equations to represent rela- 
tionships— to succeed in such tasks, 
conceptual understanding of material 
is required. Simply put, while it 
seems that both U.S. and FITW 
teachers believe conceptual under- 
standing of material is important, 
other findings suggest that more 
FITW teachers do so in practice. 

These findings indicate that both 
economic advantage and certain edu- 
cational practices contribute to math- 
ematics achievement. It seems that 
both “blessings received” and “effort 
expended” account for the superior 
performance of FITW students. 

Policy Recommendations 

In earlier publications of TIMSS 
analyses, educational researchers 
have used their best judgment to 
predict a strong correlation between 
educational practices and mathematics 
achievement. In NCREL’s June 1999 
Policy Issues, recommendations 
were made to “refocus the breadth 
and depth of the curriculum” and to 
“improve the balance between skill 
building and understanding.” Now 
with the additional analyses, educa- 
tors can support these recommenda- 
tions with sound data. 

Capitalize on Students’ Opportunity to 
Learn Key Content at an Earlier Age 

Support curricular programs that 
introduce algebra and geometry to 
more students at an earlier age. 

This study shows that FITW students 
are exposed to algebra and geometry 
at an earlier age than their U.S. 
peers. That students exposed to alge- 
bra and geometry did well on the 
TIMSS test should be expected; the 
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TIMSS test is weighted with geome- 
try and algebra items. Of particular 
significance to policymakers is that 
U.S. students might have performed 
significantly better had they also 
received algebra and geometry 
instruction at an earlier age, as did 
their FITW peers. In the end, the 
issue for policymakers should not be 
performance on the TMSS test per se, 
but rather the implications of failure— 
a constraint on students’ ability to 
compete in an international job market. 

Mathematics curricula from kinder- 
garten to eighth grade should include 
rigorous content: Pre-algebra and pre- 
geometry content should be a priority, 
along with intensive coverage of 
basic content— such as numeration, 
whole numbers, and fractions— in the 
early grades. With this approach, 
basic content can be mastered and 
then removed from the curriculum, 
giving students more time to master 
advanced concepts. One nationally 
recognized example of such a curricu- 
lar program is the Principles and 
Standards for School Mathematics 
(National Council of Teachers of 
Mathematics [NCTM], 2000). 

This type of curricular reform also 
brings with it a new philosophy and 
efficiency. Rather than spiraling con- 
tent throughout the curriculum, 
teachers and students should be held 
accountable for teaching and learn- 
ing content in a more efficient man- 
ner. As stated in NCREL’s June 1999 
Policy Issues, “less content taught 
well, and in more depth, lends itself 
to greater understanding.” (See the 
sidebar on the curriculum mapping 
Web site designed by NCREL to 
facilitate curricular reform.) 



Emphasize Conceptual 
Understanding Over Skill Building 

Support instructional standards that 
emphasize a deep conceptual under- 
standing of mathematics concepts 
rather than mere skill building. 

Early exposure to algebra and geom- 
etry is necessary but not sufficient in 
and of itself. How this material is 
taught is also significant. U.S. and 
FITW teachers alike believe that pro- 
viding deep conceptual understand- 
ing of mathematics concepts is more 
important than building skills. 
However, FITW teachers are more 
likely to put these beliefs into 
instructional practice. 




Geometry and algebra 
are taught earlier in more 
First in the World 
Consortium classrooms 
than in U.S. classrooms. 



All students need opportunities to 
think about mathematics concepts. 
They need to write about their ideas, 
justify their solutions, and explore 
strategies for solving math problems. 
FITW teachers tend to teach with 
these principles in mind: They ask 
probing questions and invite students 
to discuss the reasoning behind their 
solutions and problem-solving strate- 
gies. In other words, FITW teachers 
emphasize “comprehension over 
coverage, mastering ideas over learn- 
ing symbols, and focusing engage- 
ment and inquiry over procedure” 
(Slowinski, Laine, & van der Ploeg, 
1999, p. 7). 



NCREL 
Curricuium 
Mapping Web Site 
NCREL and the U.S. TIMSS 
National Research Center have 
launched an interactive web site 
designed to assist school districts in 
their efforts to map out new K-8 
mathematics and science curricula. 

Users are prompted to map their 
mathematics and/or science cur- 
riculum by indicating which top- 
ics are taught currently in which 
grade levels across a framework 
of 44 mathematics and 79 science 
topics. Users then are guided 
through a series of analyses, com- 
plete with a set of displays that 
superimpose their curriculum 
against that of top-achieving 
TIMSS nations. 

The site is available at: 
wwwmcreLorgIcurrmap 



Again, the Principles and Standards 
for School Mathematics (NCTM, 
2000) provides a good example. In 
addition to requiring mastery of 
algebra and geometry by eighth 
grade, these standards also list rigor- 
ous activities that require conceptual 
rather than algorithmic approaches to 
problem solving. 

Strengthen Professional 
Development for Teachers 

Invest in professional development 
for teachers that will effect conceptual 
understanding for students, improve 
standards alignment, and improve 
teachers assessment capacity 

U.S. and FITW teachers seem to 
agree on what is needed to transform 
mathematics instruction. However, 
U.S. teachers appear to lack the sup- 
port to learn how to make the trans- 
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formation happen. More intensive 
professional development is needed 
to help U.S. teachers translate their 
beliefs into practice. 

Teachers need ongoing opportunities 
to reflect on their practice, observe 
and coach colleagues, and model 
instructional techniques that empha- 
size conceptual understanding of 
math principles and strategies 
(Darling-Hammond & Falk, 1997). In 
addition, schools and school districts 
must align instruction with standards 
and appropriate assessments. Darling- 



Hammond and Falk (1997) argue that 
such systemwide re-engineering 
would help a great deal in raising 
achievement for all students. 

Teachers need to gain greater mathe- 
matics content understanding, have a 
clear and coherent standards approach 
to instruction, and develop capacity 
for using assessment tools to identify 
and correct gaps in student learning 
early on. Simply put, for U.S. schools 
to be more competitive international- 
ly, teachers need ongoing and engag- 
ing professional development oppor- 
tunities. Teachers then will be able to 
provide students with exemplary 
instruction that aligns with both stan- 
dards and assessments. This approach 
requires sustained investment by local 
and state policymakers. 
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Executive Summary 

More Than Social AdvantagCo New Evidence Links 
Curricula and Instruction to Mathematics Achievement 




The poor performance of U.S. students on the Third International Mathematics and Science Study (TIMSS) has received considerable 
attention from both educators and the general public. Also gamering attention has been the TIMSS performance of the First in the World 
Consortium (FTTW), a group of suburban Chicago school districts that independently administered the TIMSS to its students, with the test 
results ranking among the best in the world. Located along Chicago’s North Shore, Fll W communities have substantial tax bases, high 
per-capita income, and high levels of educational attainment. 

Critics often dismiss gains made by students from affluent families as a natural outgrowth of their economic advantage. The North 
Central Regional Educational Laboratory (NCREL) isolated the contribution of socioeconomic status to student achievement to allow 
for a more accurate look at the contribution of curriculum and practice. An empirical link was found between the math performance of 
Grade 8 students and more rigorous curricula and conceptual approaches to education. A full report is available at 
www.ncrel.orgirelses . 



Key Findings 
° Socioeconomic Status 

Factors associated with economic advantage, such as the num- 
ber of books in students’ home and school districts’ per-pupil 
expenditure, do account in part for FITW students’ 106-point 
math score advantage over U.S. students. But of those 106 
points, only 63 can be attributed to economic status factors. 
Such factors as school curriculum and classroom instruction 
practices appear to relate to student achievement. 

^ Curriculum 

Analysis in this study shows that eighth-grade FITW students 
are exposed to algebra and geometry at an earlier age than 
their U.S. peers. That students exposed to algebra and geome- 
try did well on the TIMSS test should be expected; the 
TIMSS test is weighted with geometry and algebra items. 

^ Instructional Practices 

Exposure to algebra and geometry by the eighth grade is not 
sufficient in and of itself: How this material is taught is also 
significant. This study found that practices that involve 
drilling students were negatively related to mathematics 
achievement while practices that promote conceptual under- 
standing of material were positively related to mathematics 
achievement. 

° Teacher Development 

The study shows that U.S. and FITW teachers alike believe 
that providing deep conceptual understanding of mathematics 
concepts is more important than simply developing skills; 
however, FITW teachers were more likely to put these beliefs 
into actual instructional practice. 

Policy Recommendations 

The key issue for policymakers should not be performance on 
the TIMSS test per se, but rather the implications of failure— a 



constraint on students’ ability to compete in an international job 
market. The following measures can help ensure that U.S. stu- 
dents will not be left behind. 

Q Capitalize on opportunities for students to learn key 
content at an earlier age 

K-8 mathematics curricula should include rigorous content. 
Pre-algebra and pre-geometry content should be a priority. 
There also needs to be intensive coverage in the early grades 
of basic content such as numeration, whole numbers, and frac- 
tions. Upon completing the eighth grade, students should be 
proficient in both algebra and geometry. 

^ Emphasize conceptual understanding over skill building 

Standards should require conceptual, rather than algorithmic, 
approaches to mathematics instruction. All students need 
opportunities to think about mathematics concepts: They need 
to write about their ideas. Justify their solutions, and explore 
strategies for solving math problems. 

^ Strengthen professional development for teachers 

U.S. teachers need ongoing and engaging professional devel- 
opment opportunities. Teachers need to gain greater mathemat- 
ics content understanding, have a clear and coherent standards 
approach to instruction, and develop capacity for using assess- 
ment tools to identify and correct gaps in student learning 
early on. Such professional development for teachers requires 
sustained investment by local and state policymakers. 

For more information, contact: 

NCR€L 

North Centnl Raglonal EducatiorMi Laboratory 
‘Applying »nd nehnology to Ltomtng" 

North Central Regional Educational Laboratory i 

1900 Spring Road, Suite 300 • Oak Brook, IL 60523-1480 
800-356-2735 • www.ncrel.org 
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^orth Central Regional Educational Laboratory 
(NCREL) is one of ten federally funded 
regional educational laboratories serving the 
needs of students and educators across the United 
States. NCREL specializes in technology integra- 
tion and serves the upper-midwestem region of 
the country. 



For over 10 years, NCREL has developed and 
delivered products and services for K-12 educators. 
For example, our Pathways to School Improvement 
Web site (vrwwjicrel.org/pathwaysJitm) receives 
over 1 ,200 visitors daily and our Learning With 
Technology course helps practicing teachers inte- 
grate technology into the curriculum in meaningful 
ways. We invite you to visit our Web site 
(wwwjicrel.org) for more information on these 
and many other resources. 



For more information on aligning standards 
and curriculum, see NCREL Critical Issue: 
Integrating Standards into the Curriculum 

ncrel.org/sdrs/areas/issues/content/ 

currclum/cu300.htm 



The recommendations in this Policy Issues address 
the rigor of mathematics curricula and 
instruction. An important starting point 
for improving education is a Oi 

carefully thought-out curriculum 
that reflects the standards 
and goals for which the 

education community is dl l' on" (I 
willing to be held 
accountable. 
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